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ABSTRACT 

Discrete  orthogonal  functions  are  used  in  adaptive  system 
identification  algorithms.  Adaptive  filters  are  realized  by 
forming  linear  combinations  of  discrete  Legendre,  Laguerre, 
and  Jacobi  polynomials,  and  backward  prediction-error 
polynomials  from  a  lattice  structure.  The  adaptive  filter 
weights  are  updated  using  the  LMS  algorithm.  FIR  and  IIR 
bandpass  filters  are  modeled  using  the  adaptive  filters,  and 
performance  comparisons  are  made. 
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I .   INTRODUCTION 

A.   ADAPTIVE  SYSTEM  IDENTIFICATION  ALGORITHMS 

Adaptive  FIR  and  IIR  digital  filters  are  widely  used  in 
system  identification  algorithms.  The  extensive  use  of  IIR 
adaptive  filters  has  been  hindered  by  several  problems: 
(1)  Instabilities  encountered  in  their  design;  (2)  Local 
minimums  in  the  mean  square  error  surface;  and  (3)  Slow 
convergence  rates  of  the  adaptive  filter  weights  [Ref.  1]. 
Although  FIR  adaptive  filters  do  not  suffer  from  the  problems 
that  ail  IIR  adaptive  filters,  they  typically  require  much 
larger  orders  to  effectively  model  a  given  system. 

Using  orthogonal  functions  in  system  identification 
algorithms  was  first  employed  by  Lee  in  1932  [Ref.  2]. 
Adaptive  filters  based  on  orthogonal  functions  have  several 
advantages,  namely,  they  are  always  stable  and  have  an 
infinite  impulse  response,  making  their  use  ideal  for  modeling 
systems  with  long  impulse  responses.  Additionally,  the  FIR 
adaptive  filter  weights  converge  faster  than  those  associated 
with  an  IIR  filter  structure  due  to  the  unimodal  mean  square 
error  surface.  The  filter  is  realized  by  forming  linear 
combinations  of  discrete  orthogonal  functions,  which  are 
weighted  to  minimize  the  mean  square  error  of  the 
approximation . 


B.   THESIS  OUTLINE 

A  brief  description  of  contents  of  the  remaining  chapters 
'follows.  Chapter  II  introduces  the  theory  of  orthogonal 
functions  and  their  use  in  modeling  linear  systems.  The 
definitions  of  the  orthogonal  functions  included  in  this 
research  are  given.  Chapter  III  develops  the  actual  adaptive 
filter  model  that  utilizes  the  orthogonal  functions  for  system 
identification.  The  derivation  of  the  filter  weights  is 
discussed  in  detail;  significant  development  of  the  discrete 
orthogonal  functions  is  shown.  Chapter  IV  presents  the 
simulation  results  of  the  various  filters  developed  in  chapter 
III  and  comparisons  are  made  between  Legendre,  Laguerre, 
Jacobi,  and  backward  prediction-error  adaptive  digital 
filters.  Chapter  V  presents  conclusions  including  limitations 
of  the  orthogonal  polynomial  filters  and  recommendations  for 
further  research. 


II.   ORTHOGONAL  FUNCTIONS 

A.   THEORY  OF  ORTHOGONAL  FUNCTIONS 

Let  {w,,  ( r )  ,  w2(  r )  ,  .  .  .  }  denote  a  set  of  real  and  continuous 
functions.  Then  the  system  of  functions  is  said  to  be 
orthogonal    in  the  range  (a,b)  if 

r  ,  f  0   f OI  m  *  n  / 1  v 

J  *.<*>"fe<*>  d^\k2n      form  =  n,  (1) 

where  kn  is  called  the  norm  of  the  corresponding  function 
[Ref.  3].   The  orthogonal  set  {wn(r)}  is  considered  complete 
if  either  of  the  following  conditions  is  true  [Ref.  2]: 
(1)  There  exists  no  function  x(r)  with 


f  x2{x)    dx   <  oo  (2) 


such  that 


f  X ( x )  Wn ( T )  dx   =  0  ,  I?  =  0  ,  1,2,...   .         (3) 

a 

(2)  For  any  piecewise  continuous  function  h(r)  with 

b 

f  h2(x)    dx   <  oo  (4) 

a 

and  an  e  >  0,  there  exists  an  integer  N  and  a  polynomial 


N 


£  <V"»<T>  <5> 


n=0 


such  that 


o  N 

f   |  h(x)   -  £  cnwn(x)    |2  dx  <  e     .  (6) 


Any  stable  causal  system,  h(r) ,  satisfies  (4)  in  the 
interval  [0,  °o)  and  can  be  represented  by  a  complete  set  of 
orthogonal  functions  [Ref.  2].  Letting  {wn(r)}  represent  a 
complete  set  of  orthogonal  functions  in  the  interval  [0,°o), 
then 


h(x)    =  { 


E  cBwn(x)     for   0  <.  x  <.   oo  ^7j 

n=0 

0         else  , 


where  the  cn  represent  the  expansion  coefficients. 

Albeit  impossible  to  form  an  infinite  sum  of  orthogonal 
functions,  it  is  practical  to  form  a  finite  sum  of  orthogonal 
functions  with  an  accompanying  error,  e,  as  given  by  (6).  It 
is  therefore  possible  to  form  an  approximate  synthesis  of  a 
linear  system,  h(r),  by  forming  finite  linear  combinations  of 
orthogonal  functions: 

£{X)     =   I  E  CnWn^)       for    0    Z    X    £°o  (8) 

0         else     . 

This,  in  itself,  is  not  particularly  significant,  for 
there  are  other  families  of  functions  that  are  not  orthogonal 


that  satisfy  (8).  A  Taylor  series  expansion,  for  instance,  is 
never  orthogonal  on  any  interval,  but  it  is  often  effective 
when  approximating  functions  [Ref.  3].  Orthogonal  functions, 
however,  have  several  desirable  characteristics  that  make 
their  use  uniquely  advantageous  when  synthesizing  linear 
systems . 

Whereas  (8)  is  based  on  a  continuous  set  of  orthogonal 
functions,  it  is  also  possible  to  form  an  approximate 
synthesis  of  a  discrete  linear  system,  n(k),  such  that 


N 


£(k)    =   £  cnwn{k)    ,£  =  0,1,2,...,  (9) 


n=0 


where  {wn(k)}  represents  a  complete  set  of  discrete  orthogonal 
functions,  and  the  expansion  coefficients,  cn,  are  chosen  to 
minimize  the  mean  square  value  of  the  approximation  error. 

B.   CLASSICAL  ORTHOGONAL  POLYNOMIALS 

The  classical  orthogonal  polynomials  form  a  subset  of 
orthogonal  functions.  The  three  classical  orthogonal 
polynomial  families  that  are  included  in  this  research  are  the 
Jacobi,  Legendre,  and  Laguerre.  Other  families  such  as  the 
Chebyshev  and  Hermite  were  found  to  be  unsuitable  for  linear 
system  synthesis  using  the  methods  described  herein. 

The  Legendre  polynomials,  denoted  by  pn("c)  [Ref.  4],  are 
orthogonal  on  the  interval  [-1,1],  and,  in  the  form  of  (1), 
their  norm  is  given  by 


1 


/  I  p"(x)   |2  dt  "  WVt  ■  <10> 


2/3  +  1 

-1 

The  Jacobi  polynomials,  denoted  by  pn(a,R)(r)  [Ref.  4],  are  also 
orthogonal  on  the  interval  [-1,1]  and  their  norm  is  given  by 

i 

f     (1  -  X)'(l  +  X)P  |  Pn(a'P)(T)  |2  C?X 
Jl  (11) 

2a+P+1  r(n+a+l)  T(n  +  P+1) 
(2n  +  a  +  P+l)  n\   T{n+OL+$+l) 

Note  that  if  a  substitution  is  made  into  (11)  with  both  or  =  0 
and  3=0,  the  result  is  equivalent  to  (10).  Therefore,  the 
Legendre  polynomials  form  a  subset  of  the  Jacobi  polynomials. 
Given  the  orthogonality  interval  of  the  Legendre  and  Jacobi 
polynomials,  [-1,1],  it  is  not  possible  to  make  a  direct 
substitution  of  the  polynomials  into  (7).  The  desired 
orthogonality  interval  for  synthesizing  causal  linear  systems 
is  [0,oo);  thus,  the  orthogonality  interval  of  the  Jacobi  and 
Legendre  polynomials  must  be  shifted  by  means  of  a  change  of 
variables.  Details  of  this  process  are  discussed  in 
chapter  III. 

The  Laguerre  polynomials,  ln(r)  [Ref.  2],  are  orthogonal 
in  the  interval  [0,00),  making  them  more  readily  applied  to  the 
synthesis  of  linear  systems  than  Jacobi  and  Legendre 
polynomials.   Their  norm  is  given  by 


f   I  la(x)  |2  dx  =  1  .  (12) 


C.   ORTHOGONALITY  OF  THE  LATTICE  PREDICTOR 

Assuming  a  stationary  input  of  random  variables,  each 
stage  of  the  lattice  predictor  is  known  to  produce  a  sequence 
of  uncorrelated  random  variables  in  the  form  of  the  backward 
prediction  errors,  {b0(k),  b.,(k),  •••}  [Ref.  3].  These 
backward  errors  are  orthogonal  in  the  range  [0,°°)  and, 
therefore,  well  suited  for  synthesizing  discrete  linear 
systems  of  the  form 

h(k)     =  \   E  CrPn^)       for    0    ±   k   <    °°  (13) 

0         else    . 

The  lattice  filter  structure  is  a  manifestation  of  the 
Gram-Schmidt  orthogonalization  procedure  insofar  as  the 
generation  of  the  backward  errors  is  concerned.  The  forward 
prediction  errors  associated  with  prediction-error  filters  are 
also  produced  by  each  stage  of  the  lattice  filter.  However, 
their  application  to  linear  system  synthesis  is  not  germane, 
for  the  forward  errors  are  correlated  and,  therefore,  not 
orthogonal  [Ref.  3]. 


III.  SYSTEM  IDENTIFICATION  AND  MODELING 

A.   ORTHOGONAL  ADAPTIVE  FILTER  MODEL 

Consider  a  system  where  x(k)  denotes  an  input  to  both  a 
causal  linear  system,  h(k),  and  an  adaptive  filter  model,  h(k) 
as  shown  in  Figure  3.1.  Let  d(k)  be  the  desired  output  of  the 
system  and  y(k)  be  the  output  of  the  adaptive  filter. 


x(k) 


System 


Adaptive 
Filter 


d(k) 


(3> 


y(k) 


e(k) 


Figure  3.1:   System  identification  configuration 

Following  the  derivation  in  [Ref.  1],  the  output  error  is 
given  by 


8 


e(k)    =  d(k)    -  yik) 

k 


=  d(k)    -   Y,    x(i)   £(*  "  i)   ■ 


(14) 


From  (9),  we  write 


N 


(k)    =  d(k)    -  £  x(i)  £  c^(/c  -  i)  ,       (15) 


j  =  -<x>        n=0 


where  {wn(k)}  represents  a  complete  set  of  discrete  orthogonal 
functions,  and  the  cn  are  the  expansion  coefficients. 
Rearranging  the  summations  in  (15)  yields 


N 


(k)   =  d(k)   -  Y,    cn   £  x(i)wn(k  -  i)     .       (16) 


n  =  0  i  =  -oo 


And  therefore, 


N 


e(k)    =  d(k)    -  Y    cnun^k)    =  d(k)-cTu(k)    ,       (17) 


n  =  0 


where 


c  =  [c0,  clt   •»,  cN]T  ,  (18) 

u(k)   =   [uQ(k),   u^k)  ,  -,   uN(k)]T  ,  (19) 


and 


un(k)    =   £  x(i)wn{k  -  i)      .  (20) 


J  =  -0O 


Figure  3.2  depicts  the  generic  orthogonal  function  model. 
We  consider  each  Qn(z)  to  be  a  black  box  that  has  two  outputs 
when  excited  by  an  impulse:   One  is  an  orthogonal  function 

9 


un(k),  and  the  other  is  a  connection  to  the  next  black  box. 
For  the  purposes  of  this  thesis,  we  limit  our  investigation  to 
Legendre,  Jacobi,  Laguerre,  and  backward  prediction-error 
functions  to  model  systems. 


x(k) 


Figure  3.2:    Generic  adaptive  filter  configuration  for 
orthogonal  functions . 

We  wish  to  find  a  set  of  expansion  coefficients  to 
minimize  the  mean  square  error  of  (17).  Using  the  LMS 
algorithm,  we  take  e2(k)  to  be  an  estimate  of  the 
instantaneous  mean  square  error  [Ref.  6].  To  obtain  the 
minimum  mean  square  error,  we  find  the  corresponding  gradient 
estimate  by  taking  the  derivatives  of  e2(k)  with  respect  to 
the  expansion  coefficients: 
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V(ic)  =  ae'(Jc)  =  2e(k)   6e(Jc)  =  -2e(k)u(k)      .  (21) 

dc  oc 

Applying  the  method  of  steepest  descent,  the  LMS  algorithm 
updates  the  expansion  coefficients  using 

er(Jc+l)  =  c(k)    +  n(-V)  ,  (22) 

where  \i    is  a  constant  that  regulates  the  convergence  rate. 
Substituting  (21)  into  (22),  we  obtain  the  LMS  algorithm: 

cr(ic+l)  =  c(k)    +2\xe(k)u(k)      .  (23) 

The  expansion  coefficients  of  vector  c  converge  in  the 
mean  when  [Ref.  6] 

0  <  |jl  <L .  (24) 

\u(k)  \2 

All  simulations  in  this  thesis  set  jj  according  to  the 
range  specified  in  (24).  The  expansion  coefficients  are 
updated  after  each  iteration  in  accordance  with  (23). 
Convergence  rates  for  the  expansion  coefficients  vary 
depending  on  model  type  and  order. 

A.   LEGENDRE  POLYNOMIALS 

Recall  from  chapter  II  that  continuous  Legendre 
polynomials  form  a  complete  set  of  orthogonal  functions  in  the 
range  [-1,1]  and  are  defined  as 


11 


/p.(T>p,,(T)    A-     1FTT      *°rm~n  (25) 

-i  [        0  for  m  *  n     . 

Following  the  derivation  given  by  Lee  [Ref.  2],  a  change 
of  variable  is  made  to  transform  the  orthogonality  range  for 
the  Legendre  polynomials  from  [-1,1]  to  [0,00),  which  is  the 
desired  range  to  correspond  to  the  causal  time  axis.  Letting 
the  first  change  of  variable  be 

t  =  2y  -  1  (26) 

causes  (25)  to  become 

1  i        2 

2  f  pm(2y  -  l)pn(2y  -  1)  dy  =  \  2n   +  1  forrn  =  n         (27) 
o  [   0     for  m  *  n     . 


Letting  the  second  change  of  variable  in  (27)  be 


y  =  e"ct 


where  C  is  any  positive  real  constant,  yields 


f  Ce-ctpm{2Ce~ct  -   l)pn(2e"ct  -  1)  dt 

for  m   =  n 


o 

1 


2n  +   1 

0     for  m  *  n     . 


(28) 


(29) 


Now,  defining 

vn(t)    =  yfC  e("c/2)t  pn(2e~ct  -   1)  (30) 

and  substituting  into  (29)  gives 


12 


/ 


vm(t)vn(t)    dt  =  { 


for  m   =  n 

2n+   1 

0     for  m  *  n 


(31) 


Since  (31)  satisfies  the  definition  of  orthogonality  given  by 
(1),  then  the  set  {vn(t)},  a  shifted  version  of  the  Legendre 
polynomials,  is  an  orthogonal  set  defined  in  the  desired  range 
[0,0°).  From  (7),  h(r)  can  now  be  represented  by  the  series 
given  by 


h{x)    =  |E  cnVn^)     for  0  <;  t  <. 

else  , 


00 


n=0 

0 


(32) 


where,  again,  cn  are  the  expansion  coefficients  chosen  to 
minimize  the  mean  square  error,  and  {vn(t)}  is  the  orthogonal 
polynomial  set  based  on  the  shifted  Legendre  polynomials. 

In  order  to  realize  a  digital  network,  it  is  necessary  to 
generate  the  shifted  discrete  Legendre  polynomials  based  on 
the  continuous  set  {v  (t)}.  It  is  convenient  to  note  the 
first  few  terms  of  the  Legendre  polynomials  [Ref.  7],  {pn(r)}: 


Po<*)  =  1 
Pi(x)  =  x 

P2(T)   -  4^2 


1 

2 


(33) 


Substituting  (33)  into  (30),  the  first  few  terms  of  {vn(t)} 
are 


v0(t)  =  /Ce~(c/2)t 

vl(t)    =  y/C(-l   +  2e~ct)e-{c/2)t 

v2(t)    =   /C(l  -  6e"ct  +  6e-2Ct)e-{c/2)t 


(34) 
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Taking  the  Laplace  transform  of  (34)  generates  the  first  few 
terras  of  the  shifted  Legendre  polynomials  in  the  frequency 
domain,  which  are  given  by 

V0(s)    =  /C 


(fir  +  C/2) 

V,(S)      -J3C- (f  -C/2)   ,    ,  (35> 

1  V     (s   +  C/2)  (s  +   3 C/2) 

V2(S)=J5C- (s  -  C/2)  (S-3C/2) 

2  V     (s  +  C/2)  (s   +  3 C/2)  (S  +  5C/2) 

From  (35),  it  is  apparent  that  the  general  expression  for 
{Vn(s)}  is 

v  (g)        A       (s  -  C/2)  (s  -   3C/2)-(s  -  (221  -  l)C/2)     r36. 
n  n     (s  +  C/2)  (s  +  3C/2)-(s   +  (2n   +  1)  C/2)  '   l   ' 

for  n  =  0,  1,  2,  ...,  where 

An  =  J(2n   +  l)C     .  (37) 

Of  the  several  techniques  available  for  performing  an 
analog  to  digital  filter  transformation,  the  matched  Z- 
transform  is  best  suited  for  our  purposes.  This  technique 
preserves  the  ability  to  express  the  digital  frequency  terms, 
{vn(z)}/  in  a  closed  form  as  in  (36)  and,  more  importantly, 
allows  for  a  filter  structure  that  is  easily  synthesized.  The 
matched  Z-transform  maps  the  poles  and  zeros  of  {Vn(s)}  into 
those  of  {Vn(z)}  through  the  substitution 

(fir  +  a)  -  (1  -  e~aT  z"1)  ,  (38) 

where  T  represents  the  sampling  period  of  the  discrete-time 
filter  [Ref.  8]. 
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Accordingly,  performing  an  analog  to  digital 
transformation  on  (36)  yields  the  first  three  terms  of 
(Vn(2)}: 

v0(z)    =    -s£- 


C/2  ™  -1 


1  -  e"-"z 

V,  (z)   =  yTc  (1  -  eC2z~1) ,39) 

1      (1  -  e-^z-1)   (1  -  e-3c'2z-x) 
V  (z)  =    v^C  (J-  -  e^z"1)  (1  -  e3^2*"1) 


(l-e-^z"1)  (l-e-30/^-1)  (l-e-507^-1) 

As  noted,  the  matched  Z-transform  preserves  the  ability  to 
express  the  terms  of  {Vn(z)}  in  the  closed  form 

-.    An   (1  -  ec/2z^)  (l  -  e3^^-1)-^!  -  e»a-i>c/az-i)  (40) 

fl  Z     (1  -  e-^z"1)  (1  -  e^^z-^-d  -  e-(2^i)c/2z-i)  ' 

for  n  =  0,  1,  2,  ...,  where 

An  =  J(2n   +  1)C  .  (41) 

The  synthesis  of  a  discrete  system  using  shifted  Legendre 
polynomials  is  accomplished  by  generating  each  Legendre 
polynomial  using  its  corresponding  transfer  function  in  (40). 
Accordingly,  substituting  each  Qn(z)  shown  in  Figure  3.3  into 
the  filter  structure  shown  in  Figure  3.2  provides  the 
necessary  filter  structure  to  generate  each  polynomial. 

C.   JACOB I  POLYNOMIALS 

The  Legendre  polynomials  form  a  subset  of  the  much  larger 
class  of  Jacobi  polynomials.  Like  the  Legendre  polynomials, 
the  Jacobi  polynomials  form  a  complete  set  of  orthogonal 
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On(z) 


M/ 


]_  _  e-(2n»IIC/2z-l 


1  -  g  (2n-l)C/2z-l 
1  -  g-(2n-l)C/2z-l 


\J/ 


Figure  3.3:   Module  of  Legendre  polynomial  adaptive  filter. 

functions  in  the  range  [-1,1]  and,  therefore,  require  a 
transformation  to  shift  the  orthogonality  range  to  [0,<»). 
The  Jacobi  polynomials,  {pn(ar,R)  ( r)  } ,  are  defined  as 


f     (1  -  x)«(l  +  T)P  Pffl(a'P)  (T)pia'P)(T)  dT  = 


^  for  m  =  n   (42) 

0   for  m  *  n  , 


where 


k2        2tt*'3*1  r(/2+a  +  P+l)   r(n+(3+l) 
*         (2r2+a  +  p+l)   n!   r(ii+a  +  P+l) 

Letting  the  first  change  of  variable  be 


(43) 


t  =  2y  -1 


(44) 


causes  (42)  to  become 
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2  /  (2  -  2y)a(2y)P  pja'P)  (2y  -  1)  pn(a'p)  (2y  -  1)  dy 

o  (45) 

{ic^  if  m  =  n 
0  if  m  *  n 

Letting  the  second  change  of  variable  in  (45)  be 

y  =  e~ct   ,  (46) 

where  C  is  any  positive  real  constant,  yields 


2  J  Ce-Ct(2  -   2e-")a(2e-ct)P  pja'p)  (2e"ct  -  1) 

o 

Zp,u'p)(2e-«  -  1)  dt   .(*"  for  in -n 

[  0  for  m  *  n  . 


(47) 


Now,  defining 

vja'p)(t)  =v^C2P/2  (2  -  2"ct)a/2  e"c(P+1,t/2  Pj;a'p)(2e-Ct  -  1)  (,48) 
and  substituting  into  (47)  gives 

f  vlm'*)(t)vim'*)(t)   dt  =  lk°     *>rm-n  (49) 

J  {    0  for  m  *  n  , 

where   kn2  is  defined  as  in  (43).   Since  (49)  satisfies  the 
definition  of  orthogonality  given  in  (1),  the  shifted  Jacobi 
polynomials,  { vn(a'R)  ( t ) } ,  form  an  orthogonal  set  of  polynomials 
in  the  range  [0,°°).   From  (7)  ,  any  causal  system,  h(t),  may 
be  represented  by 
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A(t)=lEci.v"M1(t)   if  or  0  ^  t  ^  oo         (50) 
0  else     . 

In  order  to  synthesize  discrete  linear  systems,  it  is 
necessary  to  generate  the  shifted  discrete  Jacobi  polynomials. 
Let  the  derivation  of  the  shifted  discrete  Jacobi  polynomials 
begin  by  listing  the  first  few  terms  of  the  continuous  Jacobi 
polynomials : 

Po(a'P)(t)  =  1 

Pl(a'p)(t)  =(a+l)  +  -|(a+p+2)  (t-1) 

p2(a'P)(t)  =  -|  (o+l)  (a+2)  +  — (a+P+3) (a+2) (t-1) 

+  _|(a  +  p+3)  (a+P+4)  (t-l)2)   . 

Substituting  (51)  into  (48)  yields  the  first  few  terms  of 
{vn(a'fi)(t)}: 

v0(a'P)(t)    =  J2C2*'2  e-c(P+1)t/2    (2   -  2e"ct)a/2 
vx(a'P)(t)    =  /2C2P/2  e-c(P+1)t/2    (2   -  2e-cc)a/2 

X   [(o+l)    +  —  (a  +  P+2)  (2'ct  -  2)] 
2 

v2(a'P)(t)    =v/2C2P/2  e-c(P+1)t/2    (2   -  2e~ct)a/2  (52) 

x   [(-i  (a+1)  (a+2) 

+  —  (a  +  P+3)  (  (a+2)  (2e'ct  -  2) 
2 

+  —  (a  +  P+3)  (a  +  P+4)  (2e-ct  -  2)2]      . 
8 

We  desire  to  obtain  {Vn(or,R)  (  s  ) } ,  the  freguency  domain  terms 
of  the  shifted  Jacobi  polynomials;  however,  it  is  not  possible 
to  form  a  general  expression  for  each  term  of  {Vn(a,R)(s)} 
because   the   Laplace   transform   of   (52)   yields   entirely 
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different  results  depending  on  the  value  of  a.  Therefore, 
without  losing  the  generality,  we  make  the  derivation  specific 
to  a   =   2.0  for  the  remainder  of  this  development. 

Substituting  a   =  2.0  into  (52)  and  taking  its  Laplace 
transform,  we  obtain  the  first  few  terms  of  {Vn(2,R)  ( s )  }  : 


^(2'P)(s)    = 
V2(2'P)(s)    = 


C(P+1)  }     (s+     C(p+3) 
2                                  2 
jq    (S  -    (C+D/6) 

+     C(P+1)               +     C(p+3)               +     C(P+5) 
2                                  2                                  2 
*r2    [12s2   -    (40-8)  Cs  -    (P2+6P+5)C2] 

"  (fl+  C*P+1>  )  (s+  C(P+3)  )  (s  +  C(^+5)  )  (s+  ^P^  ' 
2  2  2  2 


(53) 


where  the  K  are  constants. 

n 


Unlike  the  Legendre  polynomials,  there  is  no  apparent 
closed  form  expression  to  represent  the  terms  of  {Vn(2/fi)  ( s )  } . 
The  denominator  terms  can  be  put  into  a  closed  form 
expression,  but  not  the  numerator  terms.  This  is  an  important 
concern,  for  it  is  carried  over  to  the  frequency  terms  of  the 
discrete  shifted  Jacobi  polynomials,  {Vn(2,R)  (  z  )  }  . 

Using  the  matched  Z-transform  method  to  perform  an 
analog  to  digital  transformation  on  the  first  two  terms  in 
(53)  produces  the  first  two  terms  of  {Vn(2,B)  ( z )  } : 


(1  -  aQz^)   (1  -  a.z"1) 
W2'P)U)  =        ll 


(1  -  a^'1)   (1  -  a^-1)  (1  -  c^z"1) 


where 
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a    se^f^  (55) 


and   the   K      are   constants. 

nn 

Without  a  closed  form  representation  of  the  terms  in  (54), 
the  problem  of  generating  the  discrete  Jacobi  polynomials  is 
considerably  more  demanding  than  that  of  the  Legendre 
polynomials.  Whereas  the  transfer  function  needed  to  generate 
each  of  the  shifted  Legendre  polynomials  is  known  (see  (40)), 
the  transfer  function  needed  to  generate  each  of  the  shifted 
Jacobi  polynomials  must  be  explicitly  derived.  Due  to  the 
complexity  of  the  expressions  involved  ,  it  was  necessary  to 
use  a  symbolic  software  program  called  MACSYMA  to  generate 
both  the  terms  of  {vn(2'R)  (t )  }  and  {Vn(2'R)  ( s )  } .  The  terms  of 
{Vn(2'R)(z)}  could  then  be  found  by  transformation. 
Unfortunately,  only  the  first  15  terms  of  {Vn(a,R)(z)}  were 
derived  with  a  =  2.0  and  a  =  4.0  due  to  the  large  size  of  the 
expressions  involved. 

D.   LAGUERRE  POLYNOMIALS 

The  Laguerre  polynomials,  {ln(t)},  form  a  complete  set  of 
orthogonal  functions  in  the  range  [0,<»)  and  are  defined  by 

]laUUn(t)    dt={J  f^lH     .  (56) 

0 

Laguerre   polynomials   are   defined   over   the   desired 
orthogonality  range,  [0,<»);  thus,  we  can  immediately  proceed 
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with  the  derivation  of  the  discrete  Laguerre  polynomials  that 
are  needed  to  synthesize  discrete  linear  systems.  The  first 
few  terms  of  {ln(t)}  are: 

l0(t)    =  J2C  e~ct 

lAt)    =  J2C  (2Ct  -  l)e~ct  (57) 

J2(t)  =  v/2C(2C2t2  -  4Ct  +  l)e"ct  , 

where  C  is  any  positive  real  constant.  Taking  the  Laplace 
transform  of  (57),  we  have 

LAs)    =  v/2C 


(s  +  C) 


Ms)=^TFff^  <58) 


2 


L2{s)     =  y/2C  -15 Q 

(s  +  C)3 


Using  the  matched  Z-transform  technique  to  perform  an  analog 
to  digital  transformation  on  (58)  yields 

La(z)    =  fTC 

0  (1  -  e~cz-l) 

Li(z)  =  ^  //  "  ecZ-l?2  <59> 

(1  -  e  cz  l)2 

z     V    (1  -  e^z"1)3 
The  closed  form  general  expression  for  the  terms  in  (59)  is 

Liz)    =   v/2C   (1  "  eCz~X">n        .  (60) 

(1  -  e-cz-x)n^ 

Notice  that  all  zeros  of  (59)  are  located  at  z=ec  ,  and  all 

poles  are  located  at  z=e"c.  The  synthesis  of  any  linear  system 

using  Laguerre  polynomials  is  accomplished  by  generating  each 

Laguerre  polynomial  using  its  corresponding  transfer  function 
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given  in  (60).  Accordingly,  substituting  each  Qn(z)  in  Figure 
3.4  into  the  structure  shown  in  Figure  3.2  provides  the 
necessary  filter  structure  to  generate  the  polynomials. 


Figure  3.4:   Module  of  Laguerre  polynomial  adaptive  filter. 

E.   LATTICE  PREDICTOR 

Let  {b0(k),  b.,(k),  .  ..,  bN(k)}  denote  the  first  N+l 
backward  prediction  errors  associated  with  a  backward 
prediction  error-filter.  If  x(k)  is  a  stationary  input  of 
random  variables  to  a  backward  prediction-error  filter,  it  can 
be  shown  that  the  backward  errors  are  orthogonal,  that  is, 


E[bjk)  bjk)]    =  *«  form  =  n 
m  a  I  0  for  m  *  n  , 


(61) 


where  E[]  is  the  expectation  operator.   Following  the  proof 
given  by  Haykin  [Ref.  5],  we  write 
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bn{k)   =  £  am(m  ~  U  x^k  -  J)  '  (62) 

1=0 

where  am(l)/  1  =  0,  1,  ...,  m,  are  the  coefficients  of  a 
prediction-error  filter  of  order  m.  Substituting  (62)  into 
(61)  yields 

w         n 

E[bm(k)   bn{k)]    =  E   £  £  ajm  -  1)   an(n  -  p)  x(k  -  1)  x(k  -  p) 

1=0  p=0 
m         n 

=   £  £  ajm  -  l)an(n  -  p)   E{x(k  -  1)   x(k  -  p)] 

1  =  0     p=0 
m         n 

1=0     p=0 

(63) 

where  rx(p-l)  denotes  the  correlation  function.   The  normal 

equations  for  a  backward  prediction-error  filter  are  given  by 
[Ref.  5] 

T    an(n  -  p)   rx(p  -  1)    =  {*-  for  1  ^  n  (64) 

p&  { 0  for  1  *  n     . 

Substituting    (64)    into    (65),    we    find 


Eibjk)  bjk)]    ={$    for  mm  n  (65) 

m  n  0  for  m  *  n     . 


A  lattice  predictor  structure  is  commonly  used  to  generate 
the  backward  errors.  The  lattice  structure  shown  in  Figure 
3.5  utilizes  the  backward  errors  in  an  adaptive  filter.  Let 
the  transfer  function  needed  to  generate  each  {bn(k)}  be 
denoted  by 
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A   (*)  =  ^~ 


(66) 


By  comparing  Figure  3.5  to  Figure  3.2  we  note  that  An(z)  has 
a  definite  relationship  to  Qn(z) .  The  exact  relationship 
could  be  found  by  evaluating  each  An(z)  using  the  signal  flow 
graph  in  Figure  3.5.  Furthermore,  it  is  apparent  that 
un(k)  =  bn(k)  ,  and,  thus,  the  backward  prediction-errors  could 
be  said  to  form  a  family  of  orthogonal  polynomials.  The 
synthesis  of  a  linear  system  is  accomplished  by  forming  linear 
combinations  of  the  backward  prediction-error  polynomials. 


x(k) 


bo(k) 


Vk) 


Vi<k) 


b  (k) 

N 


N 


Figure  3.5:   Lattice  predictor  adaptive  filter. 
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IV.   SIMULATION  RESULTS 

A.   FIR  BANDPASS  FILTER  SIMULATIONS 

For  the  purpose  of  evaluating  the  FIR  system  modeling 
performance  of  the  orthogonal  polynomials,  two  bandpass  FIR 
filters  were  chosen  as  the  systems  to  be  identified.  The  FIR 
filters  were  designed  for  filter  orders  of  22  and  71,  with 
cutoff  frequencies  of  0.1  and  0.2  (fraction  of  sampling 
frequency) .  Each  filter  was  excited  by  a  zero-mean  Gaussian 
white  noise  sequence  with  unit  variance.  The  desired  output 
sequence  was  compared  against  the  output  of  the  orthogonal 
network,  and  the  error  was  used  to  update  the  expansion 
coefficients . 

Figure  4.1  shows  the  impulse  and  frequency  response  plots 
of  an  18th  order  Legendre  adaptive  digital  filter  (ADF)  with 
C  =  0.11  when  used  to  model  a  22nd  order  FIR  system.  Notice 
that  the  Legendre  filter  has  an  infinite  impulse  response  that 
allows  smaller  order  Legendre  filters  to  model  larger  order 
FIR  systems.  In  this  case  an  18th  order  model  is  used  for  a 
22nd  system,  which  is  roughly  an  18%  savings  in  terms  of  the 
filter  order.  Figure  4.2  shows  a  52nd  order  Legendre  ADF  model 
with  C  =  0.075  used  to  model  a  71st  order  FIR  system,  which  is 
approximately  a  27%  savings  in  terms  of  the  filter  order. 
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Figure  4.1:  Impulse  and  frequency  response  of  an  18th  order 
Legendre  ADF  used  to  model  a  22ndorder  FIR  filter. 
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Figure  4.2:   Impulse  and  frequency  response  of  a  52nd  order 
Legendre  ADF  used  to  model  a  71st  order  FIR  filter. 
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The  convergence  of  the  first  six  expansion  coefficients 
for  the  18th  order  Legendre  ADF  and  the  resulting  filter  output 
error  are  shown  in  Figure  4.3.  Notice  that  all  coefficients 
converge  by  the  200th  sample,  which  implies  we  need  an  input 
sequence  that  has  approximately  10  times  more  samples  than  the 
order  of  the  system  to  be  modeled. 

(a)    Convergence  of  Legendre  ADF  Coefficients 
0.2 
u 

>     0.1 
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(b)    Legendre  ADF  Output  Error 
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Figure  4.3:  (a)  Convergence  of  first  six  expansion 
coefficients  and  (b)  Output  error  for  an  18th  order  Legendre 
ADF  used  to  model  the  22nd  order  FIR  system. 

The  performance  results  when  an  18th  order  Laguerre  ADF 
with  C  =  0.98  is  used  to  model  the  22nd  order  FIR  system  is 
shown  in  Figure  4.4.  The  fact  that  the  results  are  comparable 
to  the  Legendre  ADF  is  not  surprising,  for  the  transfer 
functions  needed  to  generate  both  sets  of  polynomials  have  the 
same  number  of  poles  and  zeros  (see  (40)  and  (60)).  Likewise, 
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Figure  4.5  shows  the  result  of  a  52nd  order  model  with  C  =  0.98 
used  to  model  the  71st  order  FIR  system. 
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Figure  4.4:   Impulse  and  frequency  response  of  an  18Tn  order 
Laguerre  ADF  used  to  model  a  22nd  order  FIR  filter. 

We  expect  an  improvement  in  performance  when  using  the 
Jacobi  ADF,  for  the  transfer  function  needed  to  generate  each 
Jacobi  polynomial  has  more  poles  than  that  needed  to  generate 
the  Legendre  and  Laguerre  polynomials.  Exactly  how  many  more 
poles  are  realized  depends  on  the  value  of  a.  In  the  case 
where  a  =  4.0,  two  more  poles  are  realized  in  each  transfer 
function  that  generates  the  corresponding  Jacobi  polynomial. 
Figure  4.6  shows  the  results  of  a  15th  order  Jacobi  simulation 
with  {C,  a,  3}  =  {0.002,  4,  1000}  when  used  to  model  the  22nd 
order  FIR  system. 
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Impulse  Response:  Desired  Versus  Laguerre  ADF 
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Figure  4.5:   Impulse  and  frequency  response  of  an  52nd  order 
Laguerre  ADF  used  to  model  a  71st  order  FIR  filter. 
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Figure  4.6:   Impulse  and  frequency  response  of  a  15th  order 
Jacobi  ADF  used  to  model  a  22nd  order  FIR  filter. 
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Note  that  the  frequency  response  of  the  Jacobi  model  is 
significantly  better  in  the  lower  end  of  the  spectrum  than  in 
the  higher  end.  This  can  be  attributed  to  the  fact  that  the 
frequency  response  of  each  Jacobi  polynomial  has  lowpass 
characteristics;  therefore,  it  is  not  surprising  that  a  linear 
combination  of  Jacobi  polynomials  would  perform  better  when 
modeling  frequencies  in  the  lower  end  of  the  frequency 
spectrum.  The  same  is  true  for  both  the  Legendre  and  Laguerre 
polynomials.  The  71st  order  FIR  system  was  not  modeled  using 
the  Jacobi  polynomials  due  to  the  difficulties  encountered  in 
determining  the  transfer  functions  of  the  higher  order  Jacobi 
polynomials  as  discussed  in  part  C  of  chapter  III. 

Unlike  the  Legendre,  Laguerre,  and  Jacobi  filters,  the 
lattice  filter  structure  has  a  finite  impulse  response.  The 
15th  order  Lattice  model  produces  comparable  results  to  that 
of  the  classical  orthogonal  polynomial  models  when  used  to 
model  the  22nd  order  FIR  system  (see  Figure  4.7).  Although  the 
lattice  filter  has  a  finite  impulse  response,  Figure  4.8  shows 
that  a  52nd  order  lattice  filter  is  able  to  effectively  model 
a  71st  order  FIR  system. 
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Figure  4.7:   Impulse  and  Frequency  response  of  a  15th  order 
lattice  ADF  used  to  model  a  22nd  order  FIR  filter. 
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Figure  4.8:   Impulse  and  frequency  response  of  a  52nd  order 
lattice  ADF  used  to  model  a  71st  order  FIR  filter. 
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B.   IIR  BANDPASS  FILTER  SIMULATIONS 

As  with  the  FIR  filters,  two  Butterworth  IIR  bandpass 
filters  were  designed  to  evaluate  the  IIR  modeling  performance 
of  the  orthogonal  ADFs .  The  IIR  filters  were  designed  for 
filter  orders  of  7  and  31,  with  cutoff  frequencies  of  0.1  and 
0.2.  Since  an  Nth  order  IIR  filter  has  2N  coefficients,  our 
aim  is  to  use  orthogonal  function  adaptive  models  with 
approximately  2N  coefficients.  The  results  herein  show  the 
lowest  order  ADF  models  that  demonstrated  satisfactory 
performance. 

Figure  4.9  shows  a  15th  order  Legendre  ADF  with  C  =  0.075 
used  to  model  the  7th  order  Butterworth  IIR  filter.  Notice 
that  the  Legendre  ADF  does  not  model  the  low  order  IIR  system 
as  well  as  the  low  order  FIR  system  as  shown  in  Figure  4.1. 
The  IIR  filters  are  difficult  to  model  because  the  structure 
of  the  orthogonal  adaptive  filters  more  closely  resembles  a 
FIR  filter  (see  Figure  3.2). 

Figure  4.10  shows  a  65th  order  Legendre  ADF  with  C  =  0.055 
used  to  model  the  31st  order  IIR  filter.  Notice  that  the 
impulse  response  of  the  IIR  filter  is  still  significant  beyond 
140  samples,  but  the  impulse  response  of  the  ADF  does  a  poor 
job  of  duplicating  it. 

As  in  the  FIR  case,  the  Laguerre  ADF  performance  is 
similar  to  that  of  the  Legendre  ADF.  Figure  4.11  shows  a  15th 
order  Laguerre  ADF  with  C  =  0.98  modeling  a  7th  order  IIR 
filter.   There  is  marked  increase  in  performance  when  the 
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Figure  4.9:   Impulse  and  frequency  response  of  a  15th  order 
Legendre  ADF  used  to  model  a  7th  order  IIR  filter. 
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Figure  4.10:   Impulse  and  frequency  response  of  a  65th  order 
Legendre  ADF  used  to  model  a  31st  order  IIR  filter. 
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order  of  the  system  being  identified  is  increased. 
Figure  4.12  shows  that  the  31st  order  IIR  system  can  be  modeled 
with  a  65th  order  ADF.  The  Laguerre  ADF  has  better  performance 
characteristics  than  the  Legendre  ADF  (see  Figure  4.10)  in 
both  the  passband  and  the  stopband. 
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Figure  4.11:   Impulse  and  frequency  response  of  a  15th  order 
Laguerre  ADF  used  to  model  a  7th  order  IIR  filter. 

A  15th  order  Jacobi  adaptive  filter  with  parameters 
{C,cr,3}  =  {0.0015,  4.0,  925.0}  was  used  to  model  the  7th  order 
IIR  filter,  and  the  plots  of  the  model's  impulse  and  frequency 
response  are  shown  in  Figure  4.13.  It  appears  that  the  two 
additional  poles  in  the  Jacobi  transfer  functions  do  not 
significantly  increase  low  order  modeling  performance  when 
compared  to  the  Legendre  and  Laguerre  polynomials.  However, 
there  is  a  possibility  that  other  parameter  sets,  {C,  a,  (3}, 
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Figure  4.12:   Impulse  and  frequency  response  of  a  65th  order 
Laguerre  ADF  used  to  model  a  31st  order  IIR  filter. 
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Figure  4.13:   Impulse  and  frequency  response  of  a  15th  order 
Jacobi  ADF  used  to  model  a  7th  order  IIR  filter. 
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do  exist  that  would  produce  better  results. 

The  15th  order  lattice  ADF  simulation  of  the  7th  order  IIR 
filter  is  shown  in  Figure  4.14.  As  with  the  FIR  case,  the 
lattice  filter  performs  better  when  modeling  smaller  order 
systems.  Notice  the  poor  performance  of  the  65th  order  lattice 
ADF  when  modeling  the  31st  order  IIR  filter  as  shown  in 
Figure  4.15.   The  reason  the  lattice  filter  performs  worse 
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Figure  4.14:   Impulse  and  frequency  response  of  a  15th  order 
lattice  ADF  used  to  model  a  7™  order  IIR  filter. 

than  the  classical  orthogonal  polynomial  filters  is  that  it 
has  a  finite  impulse  response,  which  restricts  it  ability  to 
model  higher  order  IIR  systems. 
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Figure  4.15:   Impulse  and  frequency  response  of  a  65th  order 
lattice  ADF  used  to  model  a  31st  order  IIR  filter. 
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V.   CONCLUSIONS 

There  are  several  characteristics  of  orthogonal  filter 
models  that  make  their  use  attractive.  The  classical 
orthogonal  function  filters  are  particularly  adept  at  modeling 
systems  with  long  impulse  responses.  FIR  systems  with  large 
orders  (say,  greater  than  70)  can  generally  be  modeled  with 
25-30%  fewer  coefficients  using  the  orthogonal  ADFs.  Large 
order  IIR  systems  can  commonly  be  modeled  with  the  same  number 
of  coefficients  that  are  in  the  IIR  system.  The  research  in 
this  thesis  has  only  evaluated  the  performance  of  the 
orthogonal  ADF  models  when  used  to  identify  bandpass  FIR  and 
IIR  filters.  It  is  quite  possible  that  systems  with  strictly 
lowpass  or  highpass  characteristics  could  be  modeled  with 
varying  degrees  of  success.  Although  the  orthogonal  function 
filter  models  have  shown  considerable  promise,  there  are 
several  limitations  that  must  be  addressed;  recommendations 
for  future  research  are  also  presented. 

A.   LIMITATIONS  OF  ORTHOGONAL  FILTER  MODEL 

Previous  research  has  shown  that  the  convergence  rate  for 
the  LMS  expansion  coefficients  is  increased  when  using 
orthogonal  functions  [Ref.  1].  Even  so,  the  length  of  the 
input  sequence  must  be  increased  when  the  desired  filter  order 
is  increased  in  order  to  allow  the  expansion  coefficients 
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sufficient  time  to  converge.  Typically,  the  input  sequence 
must  have  a  length  that  is  10  to  15  times  greater  than  the 
desired  model  order.  This  fact  places  limitations  on  the 
orthogonal  function  models,  for  large  order  models  require 
long  input  sequences. 

The  selection  of  the  positive  real  constant,  C,  will 
significantly  influence  the  performance  of  the  Legendre, 
Laguerre,  and  Jacobi  adaptive  filters.  Furthermore,  the 
Jacobi  polynomials  have  two  additional  parameters,  a  and  (3, 
that  must  be  chosen  for  any  given  simulation.  There  is 
currently  no  known  algorithm  for  optimizing  the  selection  of 
these  parameters.  Therefore,  evaluating  the  performance  of 
the  orthogonal  function  filters  is  largely  a  matter  of  trial 
and  error. 

The  theory  for  the  classical  orthogonal  function  filters 
was  derived  in  the  continuous  time  and  frequency  domains.  The 
matched  Z-transform  technique  was  used  to  map  the  poles  and 
zeros  of  the  orthogonal  filters  from  the  s-domain  to  the  z- 
domain.  Unfortunately,  the  matched  z-transform  technique  was 
found  to  introduce  errors  concerning  the  orthogonality  of  the 
filter  output  sequences.  Other  analog  to  digital 
transformations  appear  no  more  attractive,  for  their  use  would 
eliminate  the  ability  to  express  the  filter  transfer  functions 
in  a  closed  form. 
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B.   RECOMMENDATIONS  FOR  FURTHER  RESEARCH 

An  algorithm  to  determine  the  selection  of  the  positive 
real  constant,  C,  is  needed  to  reduce  the  trail  and  error 
nature  of  the  classical  orthogonal  filter  models.  Further 
exploration  of  the  Jacobi  polynomials  is  bound  to  yield 
improved  results,  for  little  is  known  concerning  the  effect  of 
altering  a  and  (3.  The  application  of  orthogonal  filter  models 
to  a  broad  range  of  systems  should  be  explored.  Because  the 
classical  orthogonal  polynomials  are  lowpass  in  nature,  their 
application  may  be  limited. 
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